The practical cost of swapping effects

Question

I use XNA for my projects and on those forums I sometimes see references to the fact that swapping an effect for a mesh has a relatively high cost, which surprises me as I thought to swap an effect was simply a case of copying the replacement shader program to the GPU along with appropriate parameters.

I wondered if someone could explain exactly what is costly about this process? And put, if possible, 'relatively' into context?

For example say I wanted to use a short shader to help with picking, I would:

1. Change the effect on every object, calculting a unique color to identify it and providing it to the shader.
2. Draw all the objects to a render target in memory.
3. Get the color from the target and use it to look up the selected object.

What portion of the total time taken to complete that process would be spent swapping the shaders? My instincts would say that rendering the scene again, no matter how simple the shader, would be an order of magnitude slower than any other part of the process so why all the concern over effects?

Answer 1

The problem you are describing is not a "special" problem. There's nothing especially slow about changing effects on the GPU. The problem with changing effects, effect parameters (including transformations), textures, various render states, and simply sending multiple draw commands, is that it necessitates that another batch must be sent to the GPU.

Batches are CPU-bound and you only get a few thousand* per frame to use.

It depends on the CPU and what other work you are doing, but let's say you get about 1000 batches per frame. If you are rendering one object per batch, you can draw about 1000 objects on screen before you hit problems.

If you suddenly add picking, and you have to render all your objects twice, then you are limited to drawing only 500 objects.

(So if you've only got a small number of objects, then don't worry about it!)

To reduce the number of batches you are using, you basically have to "be clever" about it. The prototypical method for doing this is to cleverly combine multiple objects into a single batch. In particular, look up "instancing". Perhaps you can use the instance number to assign a unique colour to each object within your shader.

Another technique, especially suitable for picking, is to cull the rendered objects in software so that you're not rendering anything you know is not touching your picked pixel.

Here's an entire presentation slide deck from NVidia titled "Batch, Batch, Batch: What Does It Really Mean?" (PDF) that has nice graphs and stuff explaining this. It also lists some techniques for reducing your batch count.